Papers with hard data
The Unreasonable Effectiveness of Easy Training Data for Hard Tasks (2024.acl-long)
Copied to clipboard
| Challenge: | Existing pretrained language models perform well on hard data, but hard data is noisier and costlier to collect. |
| Approach: | They propose to use in-context learning, linear classifier heads, and QLoRA to show that pretrained language models generalize relatively well from easy to hard data. |
| Outcome: | The proposed model generalizes well from easy to hard data even better than oracle models finetuned on hard data. |